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(54) Copy protection for digital motion picture image data 



(57) An apparatus and method for displaying a 
copy-deterrent pattern (104) within a digital motion pic- 
ture in order to discourage recording of the motion pic- 
ture using a video camera or other sampling recording 
device. A copy-deterrent pattern (104) could be, for ex- 
ample, one or more symbols, a random pattern, a digital 
watermark or a text message (1 06). The copy-deterrent 



pattern (104) comprises a plurality of pixels within each 
frame of the digital motion picture, and the displayed pix- ' 
el intensities are modulated at a temporal frequency us- 
ing modulation characteristics deliberately selected to 
be imperceptible to human observers while simultane- 
ously producing objectionable aliasing in any copy 
made using a video camera. 
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Description 

[0001] This invention generally relates to an apparatus for displaying a time-varying copy-deterrent pattern when 
projecting a digital motion picture, the copy-deterrent pattern not visible to a viewing audience but visible in a recording 

5 of the motion picture made using a video capture device such as a video camera. 

[0002] Movie piracy is a cause of substantial revenue loss to the motion picture industry. Illegally copied movies, 
filmed during projection with video cameras or camcorders and similar devices, are a significant contributing factor to 
revenue loss. Even the questionable quality of copies pirated in this fashion does not prevent them from broad distri- 
bution in the "black market", especially in some overseas markets, and on the Internet. As video cameras improve in 

io imaging quality and become smaller and more capable, the threat of illegal copying activity becomes more menacing 
to motion picture providers. While it may not be possible to completely eliminate theft by copying, it can be advantageous 
to provide display delivery techniques that frustrate anyone who attempts to copy a motion picture using a portable 
video camera device. 

[0003] It is known to provide a distinct symbol or watermark to an original still image as a means of image or copy 

is identification, such as in order to authenticate a copy. As examples, U.S. Patent No. 5,875,249 (Mintzer et al.), U.S. 
Patent No. 6,031,914 (Tewfik et al.), U.S. Patent No. 5,912,972 (Barton), and U.S. Patent No. 5,949,885 (Leighton) 
disclose methods of applying a perceptually invisible watermark to image data as verification of authorship or ownership 
or as evidence that an image has not been altered. However, where such methods identify and validate image data, 
they provide no direct means of protection against copying an image, such as using a conventional scanner and color 

20 printer. In contrast, U.S. Patent No. 5,530,759 (Braudaway et al.) discloses providing a visible, color correct watermark 
that is generated by altering brightness characteristics but not chromaticity of specific pixels in the image. But the 
approach used in U.S. Patent No. 5,530,759 could be objectionable if used for a motion picture, since the continuing 
display of a watermark on film could annoy an audience and adversely affect the viewing experience. 
[0004] The above examples for still-frame images illustrate a key problem: an invisible watermark identifies but does 

25 not adversely affect the quality of an illegal copy, while a visible watermark can be distracting and annoying. With video 
and motion picture images, there can be yet other problems with conventional image watermarking. For example, U. 
S. Patent No. 5,960,081 (Vynne et al.) discloses applying a hidden watermark to MPEG data using motion vector data. 
But this method identifies and authenticates the original compressed data stream and would not provide identification 
for a motion picture that was copied using a camcorder. Other patents, such as U.S. Patent No. 5,809,139 (Girod et 

30 al.), U.S. Patent No. 6,069,914 (Cox), and U.S. Patent No. 6,037,984 (Isnardi et al.) disclose adding an imperceptible 
watermark directly to the discrete cosine transform (DCT) coefficients of a MPEG-compressed video signal. If such 
watermarked images are subsequently recompressed using a lossy compression method (such as by a camcorder, 
for example) or are modified by some other image processing operation, the watermark may no longer be detectable. 
[0005] The invisible watermarking schemes disclosed in the patents listed above add a watermark directly to the 

35 compressed bit stream of an image or image sequence. Alternatively, there are other watermarking schemes that add 
the watermark to the image data itself, rather than to the compressed data representation. An example of such a 
scheme is given in U.S. Patent No. 6,044,156 (Honsinger et al.), which discloses a spread spectrum technique using 
a random phase carrier. However, regardless of the specific method that is used to embed a watermark, there is always 
a concern that a watermarking method be robust, that is, able to withstand various "attacks" that can remove or alter 

40 the watermark. Some attacks may be deliberately aimed at the underlying structure of a given watermarking scheme 
and require detailed knowledge of watermarking techniques applied. However, most attack methods are less sophis- 
ticated, performing common modifications to the image such as using lossy compression, introducing lowpass filtering, 
or cropping the image, for example. Such modifications can be made when a video camera is used to capture a dis- 
played motion picture. These methods present a constant threatthat a watermark may be removed during the recording 

45 process. 

[0006] The watermarking schemes noted above are directed to copy identification, ownership, or authentication. 
However, even if a watermarking approach is robust, provides copy control management, and succeeds in Identifying 
the source of a motion picture, an invisible watermark may not be a sufficient deterrent for illegal copying. 
[0007] As an alternative to watermarking, some copy deterrent schemes used in arts other than video or movie 

so display operate by modifying a signal or inserting a different signal to degrade the quality of any illegal copies. The 
modified or inserted signal does not affect playback of a legally obtained manufactured copy, but adversely impacts 
the quality of an illegally produced copy. As one example, U.S. Patent No. 5,883,959 (Kori) discloses deliberate mod- 
ification of a burst signal to foil copying of a video. Similarly, U.S. Patent No. 6,041 ,158 (Sato) and U.S. Patent No. 
5,663,927 (Ryan) disclose modification of expected video signals in order to degrade the quality of an illegal copy. As 

55 yet another example of this principle, U.S. Patent No. 4,644,422 (Bedini) discloses adding a degrading signal to dis- 
courage copying of audio recordings. An audio signal having a frequency at and above the high threshold frequency 
range for human hearing is selectively inserted into a recording. The inserted signal is not detectable to the listener. 
However, any unauthorized attempt to copy the recording onto tape obtains a degraded copy, since the inserted audio 
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signal interacts adversely with the bias oscillator frequency of a tape recording head. 

[0008] The above-mentioned copy protection schemes disclose the use of a deliberately injected signal introduced 
in order to degrade the quality of an electronic copy. While such methods may be effective for copy protection of data 
from a tape or optical storage medium, these methods do not discourage copying of a motion picture image using a 
5 video camera. 

[0009] As a variation of the general method where a signal is Inserted that does not Impact vlewability but degrades 
copy quality, U.S. Patent No. 6,01 8,374 (Wrobleski) discloses the use of a second projector in video and motion picture 
presentation. This second projector is used to project an infrared (IR) message onto the display screen, where the 
infrared message can contain, for example, a date/time stamp, theater identifying text, or other information. The infrared 

10 message is not visible to the human eye. However, because a video camera has broader spectral sensitivity that 
includes the IR range, the message will be clearly visible in any video camera copy made from the display screen. The 
same technique can be used to distort a recorded image with an "overlaid" infrared image. While the method disclosed 
in U.S. Patent No. 6,018,374 can be effective for frustrating casual camcorder recording, the method has some draw- 
backs. A more sophisticated video camera operator could minimize the effect of a projected infrared watermark using 

is a filter designed to block infrared light. Video cameras are normally provided with some amount of IR filtering to com- 
pensate for silicon sensitivity to IR. With a focused watermark image, such as a text message projected using infrared 
light, retouching techniques could be applied to alter or remove a watermark, especially if the infrared signal can be 
located within frame coordinates and is consistent, frame to frame. A further drawback of the method disclosed in U. 
S. Patent No. 6,018,374 relates to the infrared light source itself. Since an infrared lamp can generate significant 

20 amounts of heat,' It may not be practical to project a watermark or copy deterrent image over a large area of the display 
screen using only an IR source. 

[001 0] Motion picture display and video recording standards have well-known frame-to-frame refresh rates. In stand- 
ard motion picture projection, for example, each film frame is typically displayed for a time duration of 1/24 second. 
Respective refresh rates for interlaced NTSC and PAL video recording standards are 1/60 secondhand 1/50 second. 

25 Video camera capabilities such as variable shutter speeds allow close synchronization of a video camera with film 
projection, making it easier for illegal copies to be filmed within a theater. Attempts to degrade the quality of such a 
copy include that disclosed in U.S. Patent No. 5,680,454 (Mead). U.S. Patent No. 5,680,454, which discloses use of 
a pseudo-random variation in frame rate, causing successive motion picture frames to be displayed at slightly different 
rates than nominal. Using this method, for example, frame display periods would randomly change between 1/23 and 

30 1/25 second for a nominal 1/24 second display period. Timing shifts within this range would be imperceptible to the 
human viewer, but significantly degrade the quality of any copy filmed using a video camera. Randomization, as used 
in the method of U.S. Patent No. 5,680,454, would prevent ^synchronization of the video camera to a changed display 
frequency. While the method of U.S. Patent No. 5,680,454 may degrade the image quality of a copy made by video 
camera, this method does have limitations. As noted in the disclosure of U.S. Patent No. 5,680,454, the range of frame 

35 rate variability is constrained, since the overall frame rate must track reasonably closely with accompanying audio. 
Also, such a method does not provide a mechanism for including any type of spatial pattern or watermark in each 
frame, which could be used to provide a human-readable warning message or to trace the individual copy of the film 
that was illegally recorded. 

[0011] U.S. Patent No. 5,959,71 7 (Chaum) also discloses a method and apparatus for copy prevention of a displayed 
40 motion picture work. The apparatus of U.S. Patent No. 5,959,71 7 includes a film projector along with a separate video 
projector. The video projector can be used, for example, to display an identifying or cautionary message or an obscuring 
pattern that is imperceptible to human viewers but can be recorded using a video camera. Alternately, the video camera 
may even display part of the motion picture content itself. By controlling the timing of the video projector relative to film 
projector timing, a message or pattern can be made that will be recorded when using a video camera, but will be 
45 imperceptible to a viewing audience. The method of U .S . Patent No. 5,959,71 7, however, has some drawbacks. Notably, 
this method requires distribution of a motion picture In multiple parts, which greatly complicates film replication and 
distribution. Separate projectors are required for the film-based and video-based image components, adding cost and 
complexity to the system and to its operation. Image quality, particularly for large-screen environments, may not be 
optimal for video projection and alignment of both projectors to each other and to the display surface must be precisely 
so maintained. 

[0012] Conventional methods such as those described above could be adapted to provide some measure of copy 
deterrence and watermarking for digital motion pictures. However, none of the methods noted above is wholly satis- 
factory, for the reasons stated. None of the existing copy protection or watermarking methods takes advantage of key 
characteristics of the digital motion picture environment that would prevent successful recording using a video camera. 
55 [0013] While the capability for encoding "passive" invisible digital watermarks within digital images data has been 
developed, there is a need for more aggressive copy-deterrence techniques that can be embedded within digital motion 
picture data content and can take full advantage of digital projector technology. 

[0014] Image aliasing is a well-known effect that results from a difference between the scan line or frame refresh 



EP 1 237 369 A2 



rate of an electronic display or motion picture and the sampling rate of a video camera. Inherently, image aliasing 
imposes some constraints on the image quality of a video camera recording made from a display screen. Thus it is 
known that simply varying a scan or refresh rate may result in increased levels of aliasing. For example, video projectors 
from Silicon Light Machines, Sunnyvale, CA, use a high scan rate and complex segmented scanning sequence that 

5 can corrupt a video-taped copy by producing vertical black bars in the captured image. Similar effects are also observed 
when one tries to capture an image from a computer screen with a camcorder. These are the result of differences in 
scan rates between the display and the video camera systems. These techniques, however, offer a somewhat limited 
capability for protection, since scan synchronization of video camera apparatus makes it feasible to override this pro- 
tection. Moreover, aliasing caused by simple scan rate differences does not provide a suitable vehicle for display of a 

10 warning message or other pattern in a taped copy or for digital watermarking in order to identify the source of the 
original image. 

[0015] In a fully digital motion picture system, the spectral content and timing of each displayed pixel is known and 
can be controlled for each frame. While there can be a standard refresh rate for screen pixels (corresponding to the 
1/24 or 1/30 second frame rate used for motion picture film or video displays), there may be advantages in altering the 
15 conventional "frame-based" model for motion picture display. Each displayed pixel on the screen can be individually 
addressed within any frame, and its timing characteristics can be modified as needed. This capability has, however, 
not been used for displaying a copy-deterrent pattern. 

[001 6] Therefore, it can be seen that there is clearly a need for a method that allows embedding of a copy-deterrent 
pattern within motion picture content, where the content is projected from digital data. It would be most advantageous 
20 for such a pattern to be invisible to a viewer but recordable using a video camera. Further, it can be seen that there is 
a need for a method that uses the opportunity for control of timing and of individual screen pixel content that digital 
motion picture technology offers in order to discourage movie piracy using a video camera. 

[0017] With the above description in mind, it is an object of the present invention to provide a method and apparatus 
for displaying, within a projected frame of a digital motion picture, said frame comprising an array of pixels, a copy- 
25 deterrent pattern, said pattern comprising a plurality of pixels selected from said frame, said pattern not visible to a 
human viewer but perceptible when sampled and displayed using a video capture device. In a preferred embodiment, 
the pattern is modulated at a modulation rate and modulation scheme selected to maximize signal aliasing when said 
digital motion picture is sampled by said video capture device. 

[0018] It is another object of the present invention to provide a copy-deterrent projection apparatus for projecting a 
30 digital motion picture onto a display screen, said digital motion picture comprising a sequential plurality of frames, each 
of said frames comprising an array of pixels, each said pixel assigned to be projected at a predetermined intensity for 
the duration of each said frame, said apparatus comprising: 

(a) a pattern generator capable of specifying a pattern of pixels within each said frame and capable of modulating 
35 said pattern at a modulating frequency and using a modulation timing capable of being varied so as to produce a 

modulated pattern, said modulating frequency chosen for aliasing with the displayed digital motion picture when 
sampled by a video capture device, said pattern generator also capable of specifying a variable intensity level for 
each said pixel within said pattern of pixels; 

(b) an image forming assembly capable of accepting said modulated pattern specified by said pattern generator 
40 and of projecting onto said display screen, within each said frame, said modulated pattern. 

[001 9] A feature of the present invention is the deliberate use of modulation frequency and modulation timing in order 
to obtain aliasing of the projected image when sampled using a video capture device. At the same time, however, 
modulation effects are not perceptible to a human viewer. 
45 [0020] It is an advantage of the present invention that it provides an apparatus and method for obscuring an illegal 
copy of a projected digital motion picture, where said apparatus and method apply copy protection at the time of pro- 
jection. 

[0021] It is a further advantage of the present invention that it provides a method for displaying a copy-deterrent 

effect that is imperceptible to a viewing audience. 
so [0022] It is a further advantage of the present invention that it allows digital watermarking of projected digital motion 

picture frames using modulation of projected pixels at frequencies that are not perceptible to a viewing audience. 

[0023] These and other objects, features, and advantages of the present invention will become apparent to those 

skilled in the art upon a reading of the following detailed description when taken in conjunction with the drawings 

wherein there is shown and described an illustrative embodiment of the invention. 
55 [0024] While the specification concludes with claims particularly pointing out and distinctly claiming the subject matter 

of the present invention, it is believed that the invention will be better understood from the following description when 

taken in conjunction with the accompanying drawings, wherein: 
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Figure 1 is a graph showing relative sensitivity of the human eye to flicker; 

Figure 2 is a graph showing the timing arrangement used for double-shuttering with conventional film projection; 
Figure 3 is a graph showing the timing arrangement for a single pixel in digital motion picture projection; 
Figure 4 is a graph showing a time-domain representation of a raised cosine having an oscillation frequency of 10 
Hz; 

Figure 5 is a graph showing a frequency-domain representation of a visible signal, flickering at 10 Hz; 

Figure 6 is a graph showing a frequency-domain representation of a signal oscillating at 80 Hz; 

Figure 7 is a graph showing a time-domain representation of a function oscillating at 1 0 Hz and sampled at 80 Hz; 

Figure 8 is a graph showing a frequency-domain representation of the sampled function of Figure 7; 

Figure 9 is a graph showing a frequency-domain representation of the function of Figure 7 sampled at 30 Hz; 

Figure 1 0 is a graph showing a frequency-domain representation of an aliasing condition for a sampled, oscillating 

function; 

Figure 11a is a graph showing a frequency-domain representation of a sinusoidally oscillating function truncated 
to contain only 10 cycles; 

Figure 11b is a graph showing a frequency-domain representation of a sinusoidally oscillating function truncated 
to contain only 5 cycles; 

Figure 12 is a graph showing a time-domain representation of an introduced flicker effect for a pixel; 
Figure 13 is a graph showing changing pixel intensity values for n consecutive frames of a digital motion picture; 
Figure 14 is a graph showing a frequency-domain representation of the changing pixel intensity values of Figure 13; 
Figure 15 is a graph showing a frequency-domain representation of the intensity values of Figure 13, modulated; 
Figure 16 is a graph showing a frequency-domain representation of the time-varying function of Figure 13, not 
modulated, but sampled; 

Figure 17 is a graph showing a frequency-domain representation of the time-varying function of Figure 13, mod- 
ulated and sampled; 

Figure 18 is a plane view showing a digital motion picture frame having a pattern; 

Figure 19 is a plane view showing a digital motion picture frame with a message as a pattern; 

Figure 20 is a plane view showing a digital motion picture frame with multiple regions defined for modulation of 

patterns; 

Figure 21 is a flow chart showing the decision process followed for the method of the present invention. 

Figure 22 is a schematic block diagram showing major components of the copy-deterrent projection apparatus of 

the present invention; 

Figure 23 is a block diagram showing the signal conversion path for a digital projection system; 
Figure 24 is a schematic block diagram showing major components of an alternative embodiment of the copy- 
deterrent projection apparatus of the present invention; and 

Figures 25-28 are tables 2, 3, 4 and 5, respectively, as referred to in the specification. 

[0025] The present description is directed in particular to elements forming part of, or cooperating more directly with, 
apparatus in accordance with the invention It is to be understood that elements not specifically shown or described 
may take various forms well known to those skilled in the art. 

[0026] The present invention provides a method and apparatus capable of providing copy protection and watermark- 
ing for digital motion picture display. The present invention accomplishes this purpose by introducing, as part of the 
displayed images, a copy-deterrent pattern that is imperceptible to a human observer, but that is clearly perceptible 
when captured using a video camera or related image capture device that uses sampling for image capture. In order 
to adequately disclose an implementation for practice of the present invention, it is first necessary to describe specific 
boundaries within which the method and apparatus of the present invention operate. 

Sensitivity of the Human Visual System to Time- Varying Stimuli 

[0027] The first boundary of interest relates to the flicker sensitivity of the human visual system. Flicker sensitivity 
refers to the perception of a light source with time-varying intensity (e.g., a strobe light) as a steady illumination source. 
[0028] For a time-varying stimulus at a given temporal frequency and under a given set of viewing conditions, such 
as image display size and adaptation level, an average threshold amplitude can be identified at which the time-varying 
stimulus is no longer perceived as flickering (that is, the flicker-fusion threshold). Studies show that sensitivity of the 
human visual system to sinusoidal intensity oscillations decreases dramatically at higher temporal frequencies. (Ref- 
erence is made to Kelly, D. H., "Visual Responses to Time-Dependent Stimuli: Amplitude Sensitivity Measurements" 
in Journal of the Optical Society of America, Volume 51 , No. 4, p. 422 and to Kelly, D. H., "Visual Responses to Time- 
Dependent Stimuli: III Individual Variations" in Journal of the Optical Society of America, Volume 52, No. 1, p. 89). 
Referring to Fig. 1 , which shows the flicker-fusion threshold as a function of temporal frequency, human visual system 
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sensitivity to flicker is maximized near the 1 0-30 cycles/sec range, drops off rapidly at just above 30 cycles/sec, and 
continues to drop as temporal frequency increases. For temporal frequencies above a cutoff frequency, there is es- 
sentially no perception of flicker regardless of the stimulus amplitude. This cutoff frequency occurs somewhere around 
50-70 Hz for the light adaptation levels that occur in typical display systems. 

[0029] Although the flicker sensitivity results shown in Fig. 1 refer to sinusoidal intensity oscillations, it is possible to 
derive similar curves for other types of time-varying stimuli. Of particular interest are squares waves, which have distinct 
"ON" and "OFF" phases, as opposed to the continuously varying characteristics of sinusoids. For square waves, it is 
possible to compute a "duty cycle" which is the proportion of the ON interval in a full ON/OFF cycle. A typical duty cycle 
is 50%, meaning that the ON and OFF intervals are equal. However, it is also possible to create stimuli with longer 
duty cycles, so that the ON duration is greater than the OFF duration within a single cycle. In addition, other stimuli 
might have a non-zero light intensity during the OFF phase, so that the stimuli are never completely dark. In general, 
studies have shown that increasing the duty cycle and/or increasing the intensity during the OFF phase results in 
reduced flicker sensitivity. It has been demonstrated that the sensitivity variations for many different stimuli can be 
explained by considering the amplitude of the fundamental frequency component of the stimulus waveform. (Reference 
is made to Kelly, D. H., Flicker Fusion and Harmonic Analysis" in Journal of the Optical Society of America, Volume 
51, p. 917.) 

[0030] Relevant to the present invention, when a sequence of motion picture frames is displayed at a sufficiently 
high temporal frequency, a human observer does not detect flicker but instead integrates the sequence of frames to 
perceive the effect of images in smooth motion. However, video cameras do not use the same detection mechanisms 
as the human visual system. Thus, it is entirely possible for a time-varying illumination to be captured by a video camera 
while the human observer detects only a steady illumination. 

[0031] The object of the present invention is to provide, utilizing this inherent sensitivity of the human visual system 
and using the ability of a digital motion picture projection system to control timing and intensity levels at each individual 
pixel, an apparatus and method for frustrating illegal filming of a digital motion picture using a video camera. The 
present invention operates by inserting a time-varying pattern within successive projected digital motion picture frames, 
where the time-varying pattern cannot be detected by the unaided eye but is clearly visible from a video camera. 



Conventional versus Digital Motion Picture Projection 



30 [0032] Another boundary of interest relates to the nature of motion picture projection as it has evolved using film 
during the past century, and to new capabilities inherent to digital motion picture projection. It is instructive to distinguish 
the mode of operation used by display projectors for digital motion pictures from the mode used for film projectors with 
conventional motion picture films. 

[0033] A conventional film projection system consists generally of a high brightness arc lamp and a lens assembly 

35 that are used to illuminate and project film frames onto the display screen. Film frames are typically captured at 24 
frames/sec, but projection at this same rate is undesirable as a 24Hz frequency is within the region of high flicker 
sensitivity as noted in the preceding discussion. Therefore, in order to reduce the perceptible flicker of projected films, 
a technique known as double shuttering is used. Double shuttering increases the effective display rate to 48 frames/ 
sec by alternatively blocking and unblocking the projected light twice during the projection of each frame. This concept 

40 \s shown in Fig. 2 for an ideal case of two consecutive frames, n and n+1 at intensity levels l n and l n+1 , respectively. 
Double shuttering increases the presentation rate to a frequency at which flicker sensitivity is greatly reduced as com- 
pared to the sensitivity at 24 Hz, which allows a film to be viewed without flicker. It is also noted that double shuttering 
further improves the quality of the projected images by allowing a new frame to settle (during the first "OFF" time) 
before it is projected on the screen. In a practical situation, there is a finite transition time between the "ON" and "OFF" 

45 periods and therefore, the square-shaped blocks of Fig. 2 may be replaced by saw-tooth functions, for example. 

[0034] In a conventional film projection system, each film frame is illuminated by a light source that has approximately 
constant intensity across the full extent of the frame. Moreover, each frame is sequentially projected from a film reel, 
and the average illumination intensity is held constant from frame to frame, as controlled by the shutter. In contrast, 
digital motion picture display projectors are capable of controlling, for each pixel in a two-dimensional array of pixels, 

50 multiple characteristics such as intensity, color, and refresh timing. With digital motion picture projection, the "image 
frame" presented to the viewer is a projection of this two-dimensional pixel array. 

[0035] In a digitally projected movie, there is no need for shuttering. The projected frames consist of individual pixels, 
typically made up of three primary component colors (Red, Green, and Blue, abbreviated RGB) and having variable 
intensity, where frames are refreshed at regular intervals. This refresh rate may be 1/24 of a second or higher. The 
55 transition time for the display of new pixel values, indicated as a pixel transition period in Fig. 3, is short enough that 
no perceptible flicker artifacts are produced. Fig. 3 shows a typical timing arrangement for a single pixel having two 
different intensity values, ^ and l 2 for frames n and n+1 , respectively. 

[0036] Because motion pictures are typically captured at 24 frames/sec, the description that follows uses a 24 Hz 
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frame refresh rate as the fundamental rate to be used for digital motion picture projection. However, the actual refresh 
rate could vary. The present invention is capable of adaptation to any standard refresh rate selected. As mentioned, 
the object of the present invention is to provide an apparatus and method for frustrating illegal filming of a digital motion 
pictu re using a video camera, by using the ability of a digital motion picture display system to control timing and intensity 
5 levels at each individual pixel. 

Sampling of Movie Content by Video Camera 

[0037] A video camera operates by sampling a scene at regular time intervals. By sampling at a fast enough rate, a 
10 video camera can reproduce time-varying scenes with sufficient accuracy for the human visual system to perceive the 
temporally sampled data as continuous movement. 

[0038] However, the complication with video camera sampling of a motion picture is that the motion picture display 
is not truly continuous, as is noted above. Thus, attempting to capture a motion picture using a video camera introduces 
the complexity of sampling a time-varying image display using time-varying sampling apparatus. Intuitively, it can be 

is seen that some synchronization of sampling rate to refresh rate would be most likely to yield satisfactory results. 

[0039] It may be possible to adjust the sampling rate of a capturing device to provide synchronization between the 
video camera capture frequency and the motion picture projector frequency. Frame-to-frame synchronization of a video 
camera capture frequency to a motion picture projector frequency then enables illegal filming of a displayed motion 
picture with few, if any, imaging anomalies due to timing differences. The method and apparatus of the present invention 

20 js intended to prevent any type of adequate synchronization, thereby deliberately causing interference due to frequency 
differences to obscure or mark any copy of a motion picture obtained using a video camera. 

[0040] The baseline sampling rates for video cameras can vary over a range of discrete values. Typical sampling 
rates for most video cameras commercially available are in a range between 60-1 20 Hz. For example, the NTSC and 
PAL video standards, conventionally used for commercially available video cameras, use discrete rates of 50 and 60 

25 fields per second, respectively. Optionally, in some of the so-called flickerless video cameras, multiples of these base 
rates can be used, allowing higher sampling rates of 100 or 120 Hz, respectively. These rates are, in turn, easily 
convertible to the 50 and 60 fields per second replay rates that are used in most TVs and VCRs. 
[0041] It must be noted that the present invention is not constrained to any assumption of video camera sampling 
rate being at a specific value. However, for the purpose of description, a standard, discrete sampling rate within the 

30 50-120 Hz range is assumed. In subsequent description, sampling rate is represented as ^. 

[0042] With these bounds of human visual system flicker sensitivity, pixel refresh rate of the display, and video camera 
sampling rate as outlined above, it is next instructive to describe the tools and techniques used for analyzing and 
describing frequency- related phenomena in general. 

35 Time Domain and Frequency Domain 

[0043] As is well known in the signal processing arts, it is possible to describe and quantify a time-varying signal in 
either a time domain or in a frequency domain. The frequency domain is assuredly the less intuitive of the two. However, 
in order to clearly disclose the functions performed by the apparatus and method of the present invention, it is most 

40 illustrative to utilize the descriptive tools and representation of the frequency domain. (The following discussion will 
highlight those features most pertinent to description of the present invention. A more detailed theoretical description 
can be found in an upper-level undergraduate or graduate text in linear systems analysis, from which the following 
description can be derived. The nomenclature used in the subsequent description substantially follows the conventions 
used in a standard upper-level text, Linear Systems, Fourier Transforms, and Optics, by Jack D. Gaskill, published by 

45 John Wiley & Sons, New York, NY, 1978. 

[0044] The frequency domain representation is also sometimes called the signal "spectrum". Within certain con- 
straints, the mathematical transformation between the time and frequency domains is accomplished via Fourier Trans- 
formation. Using this transformation tool, the relationship between the function in time domain, f(t), and the function in 
frequency domain, F(£), may be written as: 



50 




(1) 



55 
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f(t)=]F(4)e J2 " fi dfi (2) 

5 

[0045] Here, f (t) and F(q) are referred to as Fourier Transform pairs. It can be seen from the above equations that 
Fourier transformation is reversible or invertible. In other words, if F(£) is the transform of f(t), then f(§) will be the 
Fourier transform of F(t). 

[0046] There have been a number of corresponding pairs of invertible functions derived in working between the two 
10 domains. Some of the more useful Fourier transform pairs for the present discussion are given in Table 1 . 
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Table 1. Exemplary Fourier Transform Pairs 



Tfin p nm nin 


Freauencv Domain 


l 




cos(27t^ot) 


V4[6ft^o) + 5ft+6o)] 


rect(t) 


sine® 


comb(t/b) 


|b| comb(b^) 


W>) 


|b|F(b4) 


f(t-c) 




fl(t-c)/b] 




f(t)g(t) 


Fft)*G(fi) 



Notes to Table 1: 

• b, c and £o are constants; 

• 6( ) is Dirac's delta function; 

• rect[(x-c)/b] is the rectangular function of height 1 and of width b that is centered 
around c; 



• Sinc[(x-c)/b] = 



• comb(x/b) = |b|. - rib) represents an infinite series of delta functions that are 



separated by b. 
• Convolution is denoted by "*". 
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[0047] Note that multiplication in one domain corresponds to convolution in the other domain. A shift in one domain 
corresponds to a linear phase multiplier in the other domain (e-f 2 ^ represents the phase). 

[0048] The functions listed in Table 1 above are ideal, mathematical functions. Such idealized functions are rarely 
found under actual, measured conditions. However, functions such as these are useful for modeling and for high-level 
s assessment of actual conditions, as will be apparent in subsequent description. 

Raised Cosine as Modulation Model 

[0049] It was noted above that the present invention takes advantage of differences between human eye sensitivity 
10 to a flickering pattern and video camera sensitivity to a flickering pattern. In order to describe the present invention 
clearly, it is beneficial to consider a model type of oscillation that is conceptually simple. For this purpose, the raised 
cosine function, as shown in Fig. 4, is a suitable model that illustrates how the present invention uses a modulation 
function to induce flicker in motion image sequences. Peak values of the raised cosine waveform would correspond 
to periods of highest intensity and valleys would correspond to the periods of lowest intensity on the screen. 
15 [0050] The following is the time-domain equation for a "raised cosine" function: 

f(t) = a[1+cos(2^ m t)] (3) 

20 [0051] This function has an oscillation frequency ^ and an average or DC level equal to M a". The Fourier transform 
of f(t) may be calculated as: 

f«) = 45ft) + [Stt-Sj + 6«HJ] W 

25 

[0052] Figs. 4 and 5 show f(t) and its spectrum, F(£), respectively, for a raised cosine function. For convenience, the 
constant amplitude multiplier, a, is omitted from the figures. The function, f(t), may be used to represent an intensity 
pattern on a movie screen, varying sinusoidally with respect to time. As indicated in Fig. 5, the spectrum of a raised 
cosine contains three components: the DC component, which is represented by a delta function at the origin, and two 

30 discrete frequency components at ±£ m £ m =10 Hz in Fig. 5). A region of flicker perceptibility is also indicated in Fig. 5 
by the shaded region, with a range ± ^ where ^ is the cutoff threshold frequency for flicker visibility as in Fig. 1 . As 
the example of Figs. 4 and 5 shows, a flicker frequency, approximated by a sinusoid at 10 cycles per second, or 10 
Hz, as shown in Fig. 4, has spectral frequencies ± £ m that are well within the perceptible range. 
[0053] In contrast to the conditions of Figs. 4 and 5 where intensity flicker occurs at a 10 Hz oscillation rate, Fig. 6 

35 shows what happens when the rate of intensity flicker is increased, to 80 Hz in this example. Notice that, for the 
conditions of Fig. 6, spectral frequencies ± £ m are not within the perceptible range. In contrast to what occurs under 
the conditions of Figs. 4 and 5, a viewer cannot perceive any flickering in pixel intensity if this flickering occurs at a 
rate of 80 Hz. In terms of Fig. 6, the left and right sidebands that represent spectral frequencies ± £ m have moved 
outside the perceptible range. The viewer perceives only a constant (that is, DC) intensity level. 

40 [0054] The spectrum representation shown in Figs. 5 and 6 are idealized. In practice, actual measured modulated 
signals do not exhibit a spectrum that is as easily visualized. However, as will be seen subsequently, the representation 
given in Figs. 5 and 6 is sufficiently close for illustrating actual modulation behavior. 

Sampling Frequency Considerations 

45 

[0055] As noted above, a video camera operates by periodically sampling an image, unlike the human eye. The rate 
at which this sampling occurs, that is, the sampling frequency, affects how the video camera responds to a flicker 
pattern having a specified flicker frequency. The interplay of video camera sampling frequency and display flicker 
frequency must be considered in order to make effective use of the present invention. 

50 [0056] In order to reproduce a time-varying signal, a capture system acquires samples of that signal at given instants 
in time. Intuitively, if the samples are "close enough" to each other, the time varying signal function may be reproduced 
with great accuracy. The Shannon-Whittaker sampling theorem quantifies the above statement by indicating how close 
the samples have to be in order to exactly reproduce a signal function. According to this theorem, If a function, f(t), is 
band limited (i.e., frequency components of F(£) are contained within a limited range of frequencies, ^-l), then It is only 

55 necessary to sample f (t) at t N = — intervals, or higher, in order to perfectly reproduce f(t). This theorem is sometimes 
referred to as the Nyquist Theorem and £m=^ is referred to as the Nyquist Frequency. Mathematically, sampling of a 
function in time and frequency domains can Be described as follows: 
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f s (t) = f(t) - HA s l. combCtAs) (5) 

F s (£) - F(4) * comb(^») - IU ZF{$ - ■«.) ( 6 > 

where U is the sampling interval. $,=1/1, is the sampling frequency, and "*" indicates convolution. It should be noted 
that an idealized "zero-width" sampling function. l1/y.comb(t/t,). is used in the following description to facilitate the 
understanding of the underlying concepts. In practice, rectangular functions of finite width are used for sampling. The 
ramifications of such sampling functions will be discussed below. 

[0057] According to the above equations, sampling of a function. f(t). results In the replication of its spectrum, F®. 
at intervals L. along the frequency axis. The replicates of F© are referred to as the spectral orders of f s (t), with F© 
nt.) known as the nth spectral order. As a straightforward first approximation for initial analysis, if f(t) is selected to be 
a raised cosine function, then the sampled function and its spectrum will take the following forms: 

f s (t) = a[1 + cos(2n^ ro t)]. 11/t.l. comb(tA 8 ) (7) 

) -, 

F,(£) = ZH |&|j*«-n&)+j[*(£-£. -n4,) + 6(4+4 m -n$,)]\ ■ (8) 

5 [00581 Figures 7 and 8 depict f 8 (t) and E s £) , respectively, for a sampling frequency ^ of 80 Hz. The constant mul- 
tipliers are once again omitted for convenience. Note that in Fig. 8, because of high sampling rate, all spectral orders 
other than the 0* order are outside the region of flicker visibility. In contrast, Fig. 9 shows the spectrum of the same 
raised cosine function that is sampled at a lower rate of 30 Hz. It is evident that some of the frequency components 
of the ±1* orders have moved into the region of flicker visibility when using this lower sampling rate. If the temporally 

0 sampled version of f (t) with £ s = 30 Hz were to be subsequently displayed and viewed by a human observer, extraneous 
frequency artifacts (i.e., flickers) would be observed in addition to the inherent flicker of the raised cosine at 1 0 Hz. 
It should be noted that where oscillation and sampling frequencies interact as is shown in Fig. 9. there can be some 
remedial steps taken to minimize frequency artifacts in video camera recording, such as using temporal low-pass filters, 
for example. 

is 

Aliasing 

[00591 Recalling that the purpose of the present invention is to cause frequency artifacts, it can be seen that there 
would be advantages in causing signal aliasing, where such aliasing could not then be remedied using low-pass filter 
» techniques Aliasing is often the cause of visual artifacts in the display of sampled images that contain high frequency 
components (a familiar example resulting from aliasing is the effect by which carriage or locomotive wheels appear to 
rotate backwards in early movies). 

[00601 If the sampling rate, ^, is below the Nyquist frequency, the components from higher order harmonics will 
overlap the 0»> order components. This phenomenon is known as aliasing. This can be readily visualized by examining 

45 Figs 8 9 and 10 in progression. In Fig. 8. only the fundamental or "0 order" frequency components are within the 
perceptible region; none of the +1 or -1 order components will be perceptible to a viewer. Moving from Fig. 8 to Fig. 9, 
the affect of a change in sampling frequency is illustrated. Fig. 9 shows how reducing the sampling frequency can 
cause lower orderfrequency components (shown in dashed lines) to creep toward the origin, into the perceptible region. 
In Fig 9. components from the +1 and -1 orders now are within perceptible range. 

50 [0061] Fig 10 shows an example in which a raised cosine intensity pattern has an oscillation frequency of 50 Hz. 
This results in spectral components at ± 50 Hz. As noted above, the eye does not easily perceive the flicker of this 
pattern since this oscillation frequency $ m is near the visible threshold. Now, if the displayed pattern is captured by a 
camcorder that has a sampling rate 30 Hz. aliased components from higher order harmonics now fall within the 
region of perceptibility and degrade the viewed image. This is depicted in Fig. 10 with only the 11 st order harmonics 

55 present. It is evident that the ±2 nd spectral orders will also introduce some additional frequency artifacts into the per- 
ceptible region. For clarity, however, these orders are not shown to avoid cluttering the figure. 
[0062] Aliasing occurs whenever any portion of the higher order components overlap the 0 order frequency compo- 
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nents, as is illustrated In Fig. 10. It was noted above that, given the conditions shown in Fig. 9, a temporal low-pass 
filter could be employed to isolate only those frequency components of the fundamental order. However, it must be 
pointed out that aliasing, as illustrated in Fig. 10, does not permit a straightforward remedy using filtering techniques. 
[0063] At this point, it is instructive to re-emphasize that aliasing, as is used by the present invention, occurs as a 

5 result of sampling over discrete intervals, as performed by a video camera. The human eye does not "sample" a motion 
picture image in the same manner. Thus, aliasing effects as illustrated in Fig. 1 0 occur only with respect to video camera 
sampling; a human viewer would not perceive any such aliasing effect in watching the displayed motion picture itself. 
[0064] The preceding discussion has focused on the interaction of oscillation and sampling frequencies from an 
idealized theoretical perspective. With the principles of this interaction in mind, it is now instructive to describe some 

10 more practical aspects of actual sampling conditions, in order to provide a framework for understanding how to imple- 
ment the present invention. 

Effects of a Finite Sampling Duration 

15 [0065] The above description of spectral frequencies and aliasing used an ideal "zero-width" comb sampling function 
H/ycomtyt/y as a straightforward first approximation to show the interaction of oscillation and sampling frequencies. 
The comb function is familiar to those skilled in the digital signal processing arts. In practice, however, sampling func- 
tions have a finite integration time duration. For the purpose of this disclosure, this is termed a "finite sampling duration". 
[0066] A good approximation of a practical function with some finite width is a rectangular function of width (i.e., 

20 duration) "d" that is used to sample the function f(t) at fixed intervals t^ In such a case, the sampled time-varying 
function takes the following form: 



25 



30 



45 



55 



that is, equivalent to f(t) times a sampling function. Its corresponding spectrum equivalent takes the following form: 



F 8 £)=F(£)* 



5M**)'«w^^-jj = IW- Z [Sinc{dn^ 9 yF(4 (10) 



35 [0067] Here, the result in the frequency domain can be interpreted again as series of F(£) functions that are replicated 
at intervals. But, as equation (10) above shows, the amplitude of each nth spectral order is attenuated by a constant 
value, that is, by a sinc( ) function evaluated at dn^. Thus, the amplitude of the 11 st orders is attenuated by sine 
(d£s), the amplitude of the 12 nd orders is attenuated by sinc(2d y , etc. It is evident that if d«ts (that is, the width of 
the rectangular sampling function is much smaller than the interval between samples), the attenuation due to the sine 

40 ( ) function is negligible for the first few spectral orders. However, if the value of d becomes comparable to T s , all 
spectral orders (other than the 0 th order) will undergo attenuation dictated by the sinc( ) envelope. The ramification of 
this phenomenon is reduced visibility of the higher spectral orders if they fall in the region of flicker perceptibility. 



Time-varying functions of Finite Extent 



[0068] Referring back to Figs. 4 and 5, function f(t) was represented as a raised cosine extending from negative 
infinity to positive infinity. In practice, any time-varying function f(t) must be limited to some finite time interval . In the 
case of motion pictures this finite time interval corresponds to some portion or the entire presentation length of the 
movie. This truncation can be mathematically represented by multiplying f(t) by a rectangular function for the duration 
so of interest. Here, the duration (i.e., width of the rectangle) is selected to be D=nt,„ where n is an integer typically much 
greater than 1 and tf is an arbitrary time interval, usually representing the duration of one movie frame. Then the finite 
time-varying function f f (t) and its spectrum F f (£) can be represented by the following: 
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Hi) (,2) 



where £ D =1/D. 

[0069] Note that 2£ D is the width of the main lobe of the sinc( ) function. The main lobe contains the majority of the 
energy (i e., area) of the sinc( ). Examination of the above equations reveals that if the spectral width of the sinc( ) is 
small compared to the spectral width of F(q) (which means that the width, D, of the rectangular function is large), the 

10 slnc( ) function effectively acts as a delta function. For example, the spectrum of the raised cosine intensity pattern 
shown in Fig. 5 would contain narrow sinc( ) functions in place of delta functions. In this case, the pure spectral com- 
ponents at ±^ m are replaced by broadened components at ±^ m that have the shape of a sinc( ) function. Fig. 1 1 a shows 
this broadening effect for the raised cosine function truncated to contain 10 cycles (i.e., ^=10^). Fig. 11b shows the 
spectrum for a raised cosine function truncated to contain 5 cycles (i.e., ^=5^). 

is [0070] As % D becomes comparable to the spectral extent of Fft) (i.e., ^ = ^J, the spectral components overlap. 
This has no impact on the concepts developed so far, but makes visualization of the spectrum plots somewhat difficult. 
In a practical implementation of the present invention, the target sequence of images usually contain a large number 
of frames, and the spectral broadening effects are negligible. 

20 Realistic Intensity Function 

[0071 ] Using the raised cosine function as a model, the above description illustrates basic concepts underlying pixel 
modulation, frequency sampling, and aliasing, all viewed with respect to the frequency domain. The next part of this 
description considers a more realistic time-varying pixel intensity function f(t). Referring to Fig. 13, there is shown an 
25 example intensity function for a pixel over n consecutive image frames. (Pixel transition time, assumed to be small 
relative to frame duration, is not shown.) 

[0072] For the example represented in Fig. 13, t, is the frame duration and parameters a*, a-,, a n are used to 
indicate the height of each rectangular function (i.e., the amplitude of the intensity function for duration of frames 0, 
1 , ... n, respectively). Mathematically, this train of rectangular functions may be represented as follows: 



= *iMt -'/)+ <* 2 ^ - ^ - t f )+ ... + a n *(< - » • '/)]* ^(77) (H) 

= [a o 5(0+a 1 5(f - f / )+a 2 8(f - 2- f,)+...+a„8(f - n.t f )Yh((). (15) 

45 [0073] Equations 13 and 14 are mathematically equivalent. Insight can be gained from examination of equation 14, 
which presents the time-varying function f(t) as the convolution of a series of weighted and shifted delta functions with 
a rectangular function of width V In the context of digital projection of pixels, each delta function can represent the 
pixel intensity value for the duration of one frame. Convolution with the rect() function effectively spreads (that is, 
interpolates) the value of each delta function over its frame duration, V The rect( ) function, described in this manner, 

so can be considered to be an interpolation or spread function. This description uses rectangular spread functions to 
represent projected movie pixels. It is significant to note that other forms of spread functions can be used in place of 
rectangles without compromising the validity of this analysis. Equation (15) describes the projected frame pixels in 



their most generic form. 
[0074] The Fourier transform of f (t) from Equation (1 5) may be written generally as: 

FR) + + a2 • e*" 9 fi a n ■ e ^"''W) d*) 
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and, in this particular case: 



+ a 2 -e 




(16b) 



where £ f = 1/t, and H(£) is the Fourier transform of the interpolation function h(t). For the example of Fig. 13, the 
Fourier transform of a rect() function is a sinc() function, as presented in Equation (16b) above. The term in square 
brackets is a series of weighted linear phase functions that contains both real and imaginary parts. In order to produce 
the spectrum plot of F(£), amplitude of the term in square brackets must be calculated. This will result in a complicated 
mathematical amplitude term, A(£), that is multiplied by a sinc( ) function: 



[0075] It is possible to numerically or symbolically evaluate A(£) using scientific software packages, as is well known 
in the linear systems analysis arts. Fig. 14 shows a plot of IF(£)I for the case where n=9, tf=1/24 and a„ coefficients 
that are arbitrarily selected to be: (a^ a 1t a 2 , sig, a 4 , a 5 , a^ a 7 , a 8 , ag) = (1, 0.7, 0.8, 0.7, 0.6, 0.4, 0.5, 0.3, 0.4, 0.2). 
The plot of Fig. 14 indicates that the spectrum of this signal is nearly band limited to within ±24 Hz, but the energy of 
the signal becomes negligible beyond roughly ±12 Hertz. 

[0076] For the purposes of the present invention, it is not necessary to know the exact form of IF(£)I in equation 1 7, 
as this quantity depends on varying parameters such as n and a n . Instead, it is beneficial to examine some of the 
general characteristics of F(£). One point to note is that if A£) is evaluated at different values of \ t the highest value 
for A(£) is obtained at 4=0, namely A(0) = a 0 +a.,+...+a n . Thus one property of A(£) is that its highest value occurs at 
the origin (that is, the DC value). This property plus the fact that A(q) is multiplied by a sinc( ) function, limits the spectral 
spread of IF(£)I to within the first few side lobes of the sinc( ) function. In other words, regardless of the exact form of 
A(£), the spectrum plot of F(£) is band limited and follows a sinc( ) envelope (more generally, follows the envelope of 
the Fourier transform of the spread function, h(t)). The sinc( ) envelope is shown in dashed lines in Fig. 14. 

Modulating the Projected Pixel 

[0077] There are a number of possible modulation schemes that can be employed in order to introduce a flickering 
pattern that is not perceptible to a human viewer, but that degrades a video camera copy by using the interaction of 
frequencies described here. For the pixel represented (without flicker) in Fig. 3, the basic concept for adding a flicker 
effect is illustrated in Fig. 12. Here, a frame interval of 1/24 second is further subdivided into 8 segments which are 
then used to provide flickering on/off intervals. 

[0078] It is important to note, as shown in the example of Fig. 12, that average pixel intensity must be maintained in 
whatever modulation scheme is used. For example, using the timing arrangement of Fig. 12, a pixel is on for # of the 
total time when compared against Fig. 3. This represents a 50% duty cycle. To provide the same average intensity for 
the pixel, then, the pixel intensity during an on-time must be doubled. 

[0079] The pulse-width modulation (PWM) technique shown in Fig. 12 gives a flicker frequency of 96 Hz, outside 
the perceptible range of the human eye. However, such a PWM modulated pixel appears to flicker when recorded and 
viewed from a typical video camera. 

[0080] In Fig. 12, an idealized square wave modulation scheme is used. More generally, given a modulation function 
m(t), the modulated time-varying signal can be written as: 




(17) 



f m (t)= f(t)-m(t) = {[^(O+a^Z - f,)+a28(f -2-f,)+...+a n 8(r- n f,)]*/i(0}-m(f). 



(18) 



[0081] For the time-varying function of Fig. 13, with a rectangular interpolation function, this becomes: 
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(/)= jlv(0+*i*^ O 9 ) 



10 



15 



[0082] In practice, m(t) may be a sinusoidal function (such as the raised cosine discussed earlier) or a train of square 
waves with 50% duty cycle (as depicted in Fig. 12) or some other time-varying function. In orderto simplify the analysis, 
a raised cosine modulation function is used. This modulation function is shown as dashed lines In Fig. 12. The mod- 
ulated time-varying function then becomes: 

[0083] The Fourier transform of this signal becomes: 



20 



25 



(21) 



30 [0084] The modulated spectrum, F m (£) , consists of three replicates of the unmodulated signal spectrum, F(£), given 
by equation 17 and plotted in Fig. 14. One of these replicates is located at the origin and the other two, with one-half 
the amplitude of the original spectrum, are centered respectively around ±£ m . In conventional linear systems analysis 
terminology, F(£) , (£-£ m ) and F(£+£ m ) are sometimes referred to as the main band, right sideband and left sideband, 
respectively. The plot of F m (£) is shown in Fig. 1 5 for the parameters n=9, t,=1/24, t m =1/48 and a n coefficients (a^ a n , 

35 83, a 4 , a 5 , a^ a 7 , a 8 , 89) = (1 , 0.7, 0.8, 0.7, 0.6, 0.4, 0.5, 0.3, 0.4, 0.2). 

[0085] Note that in the absence of modulation, the signal spectrum only contains the first term, F(£). In effect, by 
introducing modulation into the system, we have advantageously increased the bandwidth (that is, the spectral spread) 
of the original signal by adding the components F(£ ± £ m ). It must be stressed once again that if a high enough mod- 
ulation frequency, is selected, the left and right sidebands cannot be seen by the observer. However, the interaction 

40 of the modulation frequency with the video camera sampling frequency, t^, will result in an aliased signal that is per- 
ceptible in the video camera copy. 

Sampling the Projected Pixel 

45 [0086] As a summary and wrap-up of the theoretical material provided above, it is instructive to illustrate, for a realistic 
example, the spectral characteristics of a sampled un-modulated signal and to compare these against the spectral 
characteristics of a sampled modulated signal. 

[0087] As has been described above, if an un-modulated signal, f (t), is sampled at tg intervals (tg< tf), spectral orders 
appear in the spectrum plot. Referring to Fig. 1 6, this is shown for the time varying function of Fig. 13, using a sampling 
so frequency of 180 Hz. Mathematically, the Fourier transform of the un-modulated and sampled signal can be written 
as follows: 



55 



\f, (o| = \a{4) ■ siJ^J- j + Ms - e. ) ■ Sin \^\ + A <£ + 0 * "{^] + • 



(22) 
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[0088] If, instead of an un -modulated signal, sinusoidally modulated pixel values are sampled at a rate £ s =1/ts (as 
with a video camera, for example), the spectrum contains broader spectral orders. This is depicted in Fig. 17, using a 
sampling frequency of 180 Hertz and modulation frequency of 48 Hertz. Mathematically, the spectrum of the sinusoi- 
dally modulated and sampled signal can be represented by the following: 

5 

= F(f) + I.F(f-f m ) + I.F( f+ d 
io * i 1 

»5 +... 

(23) 

[0089] The fundamental order is given by the terms on the second line of equation 23, the ±1 orders appear in the 
20 third line, etc. According to the Nyquist Theorem, in order to produce aliasing, the sampling frequency must be less 
than twice the bandwidth of the signal. If spectral spread, W, of F(£) is taken into account, this criterion becomes ^< 
2K m +W/2). 

[0090] It is instructive to note that in the equations derived above, care has been taken to express intensity, modu- 
lation, and interpolation functions in a generic form, using f(t), m(t), and h(t). This allows these equations to be applied 
25 to any type of real-world function as it may be encountered in digital projection of frames and pixels. In addition, par- 
ametric representation of quantities such as modulation frequency £ m , sampling frequency £ s , and bandwidth W allow 
straightforward manipulation using the underlying concepts outlined above. 

[0091] Therefore, in a practical application for deliberately causing aliasing effects in digital motion picture projection, 
it is sufficient to employ best approximations of f(t), m(t), h(t), and other variables and, with the aid of linear systems 
30 and Fourier analysis tools as developed above, to determine the best conditions for maximum aliasing. 

Exemplary Tables and Calculations 

[0092] For the purpose of describing the method and apparatus of the present invention in detail, it is instructive to 
35 provide a summary listing of those variables that must be considered in order to deliberately cause aliasing artifacts 
when using a video camera to record a digital motion picture. Using the naming conventions employed in the above 
description, the following parameters must be considered: 

(a) Sampling frequency £ s . This parameter is not in the control of any copy protection method or apparatus. The 
40 method of the present invention will be effectively implemented by making some assumptions on likely sampling 

frequencies that could be used by a video camera when attempting to make a copy of a displayed motion picture. 
Standard sampling rates widely used by commercially available video cameras include 1/50 second (the standard 
NTSC rate) and 1/60 second (the standard PAL rate), and so-called "flickerless" speeds of 1/1 00 second and 1/1 20 
second. Other rates are possible; the method of the present invention be employed where image aliasing appears 
45 at the most widely used sampling frequencies as well as in cases where non-standard sampling rates are utilized. 

(b) Bandwidth, W, of the unmodulated signal or the one-sided bandwidth, W/2, as is used in some instances 
throughout this application to denote the spectral extent in either the positive or negative frequency domains. As 
an upper limit, the bandwidth of the unmodulated signal is the displayed frame rate. As noted in the above de- 
scription, the frame rate has a likely value of 24 Hz. However, the present invention can be used with other frame 

so rates. It must be noted that the "unmodulated" signal, as this term is used here, refers to continuous pixel intensity 

as is shown in Fig. 13. The actual bandwidth value is not controlled by the present invention; instead, W is de- 
pendent on the frame rate and on the scene content, frame to frame. Thus, a practical limit of bandwidth less than 
or equal to the frame rate is sufficient for purposes of the present invention. 

(c) Modulation frequency £ m . This parameter is controlled using the method of the present invention, taking into 
55 consideration parameters (a) and (b) above and also considering practical constraints imposed by intensity limi- 
tations of the projection system. For any individual pixel or set of pixels, a modulation frequency £ m is selected so 
that the effective bandwidth of the modulated signal is such that aliasing will be perceptible when sampled using 
a video camera. 
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[0093] The method of the present invention, then, consists in careful selection of a modulation frequency that is not 
visible to the unaided eye, but causes aliasing when sampled by a video camera. 

[0094] As was noted with respect to Fig. 14, a result of using a finite sampling duration is the attenuation of higher 
order spectral components. For this reason, it is generally considered most practical to consider the first order (± 1 
5 order) spectral components for creating the aliasing effects of the present invention. Second order spectral components 
may also have sufficient energy to cause visible aliasing in some instances. 

[0095] Referring to the examples illustrated by Tables 2, 3, 4, and 5, (see Figures 25-28, respectively) there are 
shown values of spectral spread for various modulation frequencies, using B flickeriess u conditions of sampling frequen- 
cy (i.e., £s=120 Hz), varied for modulated signal bandwidth. 
10 [0096] ' Recall from the above discussion that the spectral spread of the zero order modulated signal with modulation 
frequency £ m and bandwidth W ranges from -£ m + W/2) to £ m + W/2). Table 6 below summarizes the spectral spread 
values for different spectral orders. In the example of Fig. 15, £ m = 48 Hz and W/2 = 24, so that the spectral spread of 
the zero order component, according to Table 6, is -60 to +60 Hz. 
[0097] Aliasing due to the 1 st spectral order begins as soon as the following is true: 

15 

t^-(^ + W/2)]<[(^ m+ W/2)] (24) 
[0098] Solving the inequality for £ m results in the lower limit for the modulation frequency: 

20 

t.*{L£) < 25 > 

25 

[0099] Aliasing continues until the following condition holds: 

K s+ (4 m +W/2)]>.(§ m+ W/2) (26) 

30 

[0100] Solving inequality (26) for £ m results in the upper limit for the modulation frequency. Thus, the condition for 
modulation frequency to produce first order aliasing can be expressed as follows: 



[0101] Similar calculations can be carried out to determine the range of modulation frequencies that can produce 
40 aliasing for other orders. Table 7 summarizes these results for the first few orders, using the spectral spread values of 
Table 6. 



Table 6. 



Spectral spread of the fundamental, 11 st and ±2"" orders for a signal with bandwidth W and modulation 
frequency 4m- 




Spectral Range 


From 


To 


0 th order 




+Kn,+W/2) 


+1 8 * order 


S.-&.+W/2) 


k + ^+W/2) 


-1 st order 


-Vttm+W/2) 


-^ + (!j m+ W/2) 


+2 nd order 




2^ + ft m+ W/2) 


-2 nd order 




-2^+ tfj.n+W/2) 
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Table 7. 



Modulation Frequency range that can produce aliasing for different orders 


Aliasing order 


Modulation Range 


+1st order 


fe-Wy2<^ m <(^ + W)/2 


-1 st order 


(-^-Wy2<^<(^ + W)/2 


+2 nd order 


(2^ s -W)/2<^ m < 




(2^ s +W)/2 


-2 nd order 


(-2^ s -W)/2<5 m <(- 




24s+wy2 



is [0102] It must be noted that the aliasing conditions summarized in Table 7 are necessary— but not sufficient— con- 
ditions for visible aliasing. In order to obtain the necessary and sufficient conditions, it is necessary to look more closely 
at the actual frequency spread within each of the spectral orders. This is because it is unlikely that the entire spectral 
range of the modulated signal (i.e., ±(£ m +W/2)) is populated with spectral components. To illustrate this point, re- 
examine Figs. 14 and 15. If W/2 and £ m are selected to be 12 and 48 Hz respectively, the modulated signal of Fig. 15, 

20 by definition, has a bandwidth of ±(q m +W/2) = ±60 Hz, as was noted above. But, as evident from the figure, the energy 
of the spectrum in not uniformly distributed within the band of ±60 Hz. In fact, most of the energy is only concentrated 
around three spectral frequencies of 0, +48 and -48 Hz. Thus, if this signal is sampled, only the components around 
the main band, the left sideband and the right sideband of the higher orders have sufficient energy to create strong 
aliasing effects. In a practical system design, care must be taken to ensure that aliasing is caused by the portions of 

25 the spectrum with a significant amount of energy (i.e., by the main, left and right sidebands). 

[01 03] Tables 2-5 provide spectral spread calculations for different values of modulation frequencies, £ m . Entries in 
each table indicate the spectral spread of the main, left and right bands of the first and second spectral orders for the 
modulated and sampled signal. For example, in Table 2, the spectral spread of the left sideband of the +1 81 order for 
a signal with W/2=30, ^=120 and £ m =50 Hz is calculated to be from 40 to 100 Hz. The results for the corresponding 

30 negative spectral orders can be obtained by reversing the signs of the entries. Also note that aliasing only becomes 
visible if the spectral spread of these higher orders falls somewhere within the frequency range ±40 Hz. 
[01 04] Using the straightforward calculations described above, any number of tables for a given sampling frequency 
£s and bandwidth can be derived, over any number of modulation frequencies, t^. It is important to note that Tables 
2-5, each tabulated frequency spread values for a different estimate of W/2 namely, W/2 of 30, 24, 12, 5, respectively. 

35 Alternatively, an exact value of W/2 could have been obtained by examining the Fourier transform of the time-varying 
function if the intensity values of the projected pixels within the target frames were precisely known. The selection of 
suitable modulation frequencies for causing aliasing can be accomplished by inspection of such tables. 
Table 2 lists spectral spread values for 1 st and 2nd spectral orders given a one-sided bandwidth W/2 of 30 Hz, sampling 
frequency ^ of 120 Hz, for modulation frequencies from 50 Hz to 210 Hz. 

40 [0105] Table 3 lists spectral spread values for 1st and 2nd spectral orders given a one-sided bandwidth W/2 of 24 
Hz, sampling frequency ^ of 120 Hz, for modulation frequencies from 50 Hz to 210 Hz. 

[0106] Table 4 lists spectral spread values for 1st and 2nd spectral orders given a one-sided bandwidth W of 12 Hz, 
sampling frequency ^ of 120 Hz, for modulation frequencies from 50 Hz to 210 Hz. 

[0107] Table 5 lists spectral spread values for 1st and 2nd spectral orders given a one-sided bandwidth W of 5 Hz, 

45 sampling frequency ^ of 120 Hz, for modulation frequencies from 50 Hz to 210 Hz. 

[01 08] For the purpose of finding a suitable modulation frequency £ m under given conditions, the preferred approach 
is to consider worst-case conditions. Among factors to be considered is bandwidth. A narrow bandwidth is a worst- 
case condition, since it is more difficult to cause aliasing with a narrow bandwidth signal. Table 5, with values for a 
one-sided bandwidth W/2 of 5, represents worst-case conditions among Tables 2-5 . Using just one example from Table 

so 5, ft can be seen that aliasing at frequencies within the visible range can be caused using modulation frequencies £ m 
between 70 and 165 Hz. The optimum values appear in the middle of this range. For example, with a modulation 
frequency £ m of 1 1 0 Hz, the left sideband of the 1 order is centered at roughly around 1 0 Hz. Th is creates a distinctly 
visible aliasing condition, no matter how narrow the bandwidth. Other modulation frequency £ m values near this 110 
Hz frequency also are likely candidates for causing aliasing. 

55 [0109] Note that negative spectral orders (-1 order, -2 order) are not listed in Tables 2 - 5. However, these values 
can be expressed simply by changing the sign for each spectral value listed. 

[0110] Simple arithmetic calculations are all that is needed to obtain the values that populate successive rows of 
Tables 2-5. For the 1 st and 2 nd spectral orders in Tables 2-5, the location of the main band is determined by sampling 
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frequency Since sampling frequency is 120 Hz, the 1 st spectral order main band is centered at 120 Hz (£g). The 
2 nd spectral order main band is centered at 240 Hz (2^) respectively. The From/To spread of the main band and of 
both left and right side bands is set by the bandwidth W, which differs for each of Tables 2-5. The left and right side 
bands are centered using the following simple calculation: 

Center of main band - Modulation frequency £ m = Center of left band (28) 



Center of main band + Modulation frequency £ m = Center of right band (29) 

[0111] It should be noted that the previous analysis of aliasing assumed the simplest case, which is the use of a 
sinusoidal modulation frequency. In actual practice, it may prove advantageous to employ a different type of modulation 
signal waveform, as we will discuss in the next section. 

Method of Preferred Embodiment 

[0112] The present invention temporally modulates the displayed pixel intensities in such a way that objectionable 
patterns will be produced when the displayed pixel intensities are recorded with a video camcorder. More specifically, 
the displayed pixel intensities are modulated so that an observer of a displayed movie will not see any degradation in 
image quality, but any attempt to capture the displayed movie with a camcorder will result in aliased temporal frequency 
components that will be readily visible when the camcorder copy is subsequently viewed. According to the present 
invention, individual pixels or groups of pixels are modulated in various ways to produce specific spatial patterns in the 
video copy and to prevent a video pirate from circumventing the degradations that are produced by the patterns. The 
key aspects of the present invention are 1) the spatial arrangements of pixels that undergo temporal modulation; 2) 
the temporal modulation signal waveform; and 3) the temporal modulation frequency (or frequencies). We now discuss 
each of these aspects. 

[0113] Referring to Fig. 1 8, there is represented an arbitrary frame 1 00 of a digital motion picture. Frame 100 com- 
prises an array of pixels 102 that display the scene content. To display scene content in color, each individual pixel 
102 actually comprises a red pixel 102R, a green pixel 102G, and a blue pixel 102B, where the red, green, and blue 
color pixels are visually overlapped and intensities are varied using color representation techniques well known In the 
imaging arts. However, for simplicity of description, the model of a single, generic pixel 102 is used here as a gener- 
alization. The description provided here refers to individual RGB color components only when necessary. 
[0114] As Fig. 1 8 shows, a spatial pattern 1 04 of pixels 1 02 within frame 1 00 can be identified for modulation using 
the method of the present invention. There are numerous options for selecting and modulating one or more patterns 
104 during display of a digital motion picture. Chief among selection options for specifying pattern 1 04 composition 
are the following, including combinations of the following: 

(1a) random arrangement of pattern 104. In a preferred embodiment, a random selection of pixels 102 is used to 
produce a pattern 1 04 that is optimal for obscuring a video camera copy of the digital motion picture. Pattern 104 
can be changed for each frame 1 00 or for each set of n frames. The use of a random modulated pattern 1 04 that 
is constantly changing obviates the use of temporal filters or other image processing techniques for re-creating an 
acceptable image from the degraded video copy. 

(1b) arrangement of pattern 104 as a text message. In another preferred embodiment, pattern 104 could be em- 
ployed as a "bitmap" for a message 106 comprising text and symbols, as shown in Fig. 19. Message 106 could 
be the same for each frame 100, such as a message that displays the name of the theater where the projection 
apparatus is located, but concurrently part of message 106 could be changed every n frames to display time- 
varying information such as the date and time. Message 106 could also display a simple message such as "ILLE- 
GAL COPY" or "STOLEN," or a more detailed message could be displayed, such as a reward notice or other 
incentive for return of the illegal copy. 

(1c) arrangement of pattern 104 as a pictorial image, possibly as part of an animation. Pattern 104 could be em- 
bodied as an image intended to obscure or provide information. Animation techniques could be employed to gen- 
erate an informational or annoying pattern 104. 

(1 d) arrangement of pattern 1 04 as a watermark message. A watermark pattern 1 04 could comprise a plurality of 
modulated pixels 1 02 within various parts of frame 1 00. For example, an algorithm could be used for determining 
which pixels 102 are used to create a digital watermark, assigning a spatial distribution to the watermark. Crypto- 
graphic methods could be employed, in conjunction with the algorithm used for pixel 102 selection, to securely 
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encode watermark information. In order to recover a message contained within a watermark pattern 104, a de- 
cryption key may be required for deciphering contents in accordance with the watermark embedding algorithm. 
(1e) entire frame 100 considered as pattern 104. In certain implementations, it may be advantageous to modulate 
every pixel 1 02 within frame 1 00, typically with the same modulation frequency for every pixel. However, if such 
5 a method were used, it would then be preferable to change modulation frequency periodically or randomly, to 

prevent synchronization of video camera timing with modulation frequency. 

[0115] The above techniques for pattern 104 selection are just some of the more likely techniques that could be 
used, all within the scope of the present invention. 

10 [0116] There are some practical limitations that may constrain the modulation of an individual pixel within the chosen 
pattern 104. This is because the process of modulating the pixel intensity inherently results in a lower average intensity 
for a given peak intensity value. For example, if pixel 1 02 corresponds to a very bright displayed value, it may not be 
possible to modulate pixel 102 because the peak intensity that is needed to maintain the average intensity for pixel 
102 may exceed the projector capabilities. Thus, it may be necessary to modulate other nearby pixels 102 or other 

15 areas of frame 1 00, where the peak intensity requirements can be met. Thus, pixels in an image frame may be analyzed 
according to a peak intensity criterion and pixels meeting the criterion further determine the pattern that is subject to 
the temporal modulation. This criterion may be pixels not exceeding a certain brightness level. Additionally, there may 
be some types of scene content or locations within the scene for which induced flicker may be perceptible at higher 
modulation frequencies than average. This may necessitate applying modulation to other regions of a displayed frame. 

20 [0117] Some of these limitations are addressed in the present invention by choosing the proper mode for the temporal 
modulation. In a preferred embodiment, the modulation signal waveform is a sinusoid, as we described previously in 
the analysis of aliasing. A sinusoidal function will minimize the spectral extent of the frequency sidebands that are 
produced by the modulation , which makes it somewhat easier to place the sidebands at the desired frequency position. 
However, many projection systems may not be capable of modulating the pixel intensities according to a pure sinusoidal 

25 waveform, so another preferred embodiment is to approximate a sinusoidal waveform using a rectangular or sawtooth 
modulation signal waveform. In a preferred embodiment using a rectangular modulation waveform, a 50% duty cycle 
is used, which requires a doubling of the peak pixel intensity to maintain the same average intensity as an unmodulated 
signal. In still another preferred embodiment, the rectangular waveform may not be completely dark during the OFF 
period, in order to reduce the peak pixel intensities that are required to maintain a desired average intensity. This 

30 potentially allows for more pixels to be modulated when a display device has a limited maximum intensity. The tradeoff 
is that the resulting aliased components will not be as severe as a full ON/OFF modulation. Another useful variation 
of the modulation waveform is to use a duty cycle of more than 50%, which also lessens the peak intensity that is 
required to maintain the same average intensity. Any number of other possible modulation waveforms can be employed, 
ail within the scope of the present invention. 

35 [0118] In addition to the basic shape of the modulation signal waveform, there are a number of temporal modulation 
options for pattern 104 as part of the present invention. These temporal modulation options include the following, and 
combinations of the following: 

(2a) using a single modulation frequency. In a preferred embodiment, a single modulation frequency is used to 
40 modulate the pixels that comprise pattern 1 04. It is advantageous to change the modulation frequency periodically 

or randomly, in order to frustrate attempts to synchronize video camera recording equipment to the modulation 
rates in use. 

(2b) using multiple modulation frequencies. In another preferred embodiment, two or more different modulation 
frequencies are used in two or more regions 1 08 of frame 1 00, as is shown in Fig. 20. The use of different modulation 

45 frequencies within a frame 1 00 allows for video cameras with different sampling rates to be simultaneously affected 

to the maximum extent. Even with a single camera operating at a single sampling rate, the use of two or more 
modulation frequencies in different regions will introduce patterns that flicker at different rates, which can be highly 
objectionable. When multiple modulation frequencies are used within the same frame, it is extremely difficult to 
synchronize video camera recording equipment to all modulation rates simultaneously. However, it again may be 

so advantageous to change the modulation frequencies periodically or randomly, in order to further frustrate attempts 

to synchronize the video camera to the modulation rates in use. 

(2c) modulation of pixels 1 02R, 1 02G, 1 02B in pattern 1 04. Pixels in each R, G, and B color plane can be modulated 
independently of one another or synchronously, within the scope of the present invention. It may be advantageous 
to modulate pixels within only one or another color plane to simplify apparatus design, for example. 
55 (2d) using frequency modulation techniques as a form of digital watermarking. In addition to obscuring any video 

camera copy, modulation of pattern 1 04 can also be used to encode an information signal. Changes in modulation 
frequency can thereby be used as an encoding technique for copy watermarking and for information such as 
location, time and date of projection, identifying number of the film copy sent, and related information. 
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(2e) using pulse-width modulation. This variation on a frequency modulation technique uses manipulation of the 
duty cycle of pixels 102. Pulse width modulation can be particularly useful where intensity levels needed for pixel 
1 02 modulation may not be possible for a projector to achieve at 2X, as is needed for a 50% duty cycle modulation. 
Using this technique, however, the intensity must be preserved for each modulated pixel 102. 
(2f) using amplitude modulation. This modulation technique can employ the inherent capability for digital cinema 
to control intensity of each pixel 102 over a range. An amplitude modulation scheme would enable encoding of 
watermark information within a sequence of variable intensity values. 

[0119] It would also be feasible to combine or to alternate any of methods (2a) through (2f) to implement a hybrid 
modulation scheme for digital watermarking, within the scope of the present invention. 

[01 20] Once selections have been made for the spatial arrangement of pattern 1 04 (as in methods (1 a) - (1 e)) and 
the temporal modulation waveform and mode (as in methods (2a)-(2f)), it is necessary to choose the specific modulat.on 
frequency (or frequencies). In general, a reasonable design approach is to select the modulation frequency so that 
one of the first-order side bands of the sampled signal is centered in the frequency range of approximately 10 to 30 
Hz This produces an aliased component in the sampled signal that is in the peak sensitivity range of the human visual 
system as shown in Fig. 1 . From equations 28 and 29, we can see that this design approach is equivalent to satisfying 
the following equation: 



10 Hz < (Sampling frequency $ s - Modulation frequency £ m I < 30 Hz. 



(30) 



For example if the sampling frequency ^ of the specified camcoder is 120 Hz, then a modulation frequency of either 
1 00 Hz or 1 40 Hz will place the center of a first-order side band at 20 Hz, thus producing a strong aliased component 
in the sampled video signal. Since many camcorders use a sampling frequency of either 60 Hz or 1 20 Hz, the selection 
of a modulation frequency at 90 Hz will produce an aliased component at 30 Hz for either sampling frequency. In this 
way a single modulation frequency can produce the desired effect regardless of the particular camcorder that is used. 
However as described in methods (2a) - (2b), it may be advantageous to use different modulation frequencies, either 
for different regions within a single frame or for a given region across multiple frames, to provide an even greater 
deterrent to video piracy. For example, using modulation frequencies of 80 Hz and 100 Hz for different regions in a 
frame 100 will produce aliased components at 20 Hz and 40 Hz regardless of whether the sampling frequency is 60 
Hz or 120 Hz In this way, there is always an aliased component in the camcorder video at 20 Hz, which is near the 
peak flicker sensitivity, thus producing a highly objectionable pattern. In other applications, it may desirable to select 
a modulation frequency that moves the aliased component to very low temporal frequencies, say less than 10 Hz. The 
visual appearance of a slowly varying pattern may not be as visually objectionable as a rapidly varying pattern, but if 
pattern 1 04 represents a text message, the slowly varying pattern may be more easily comprehended. Finally, when 
changing the modulation frequency (or frequencies) periodically or randonly to prevent the synchronization of video 
recording equipment, the amount of change from the preferred frequency (orf requencies) does not need to be extremely 
large. Even small changes, such as ± 5Hz, will be sufficient to prevent continuous synchronization. 

Summary of steps for implementation of method 

[0121] The basic steps for implementing the method of the present invention are as follows: 

1 Identify pattern 1 04 to be used. Decide upon a strategy for displaying pattern 1 04, using options (1 a)-(1 e) given 
above, or some other technique for pattern 104. This decision depends, in large part, on the purpose for which 
this invention is to be applied. For example, when maximally obscuring any copied movie content is the goal, as 
in the preferred mode of the present invention, a randomly changing pattern 104 has advantages. 
2. Select the appropriate mode for temporal modulation. Options (2a)-(2f) given above provide the preferred options 
for temporal modulation available in implementing the present invention. 

3 Select the appropriate modulation frequency or frequencies. Suitable modulation frequencies can be identified 
using the techniques described above and those used to generate Tables 2-5. As was noted, the best option is 
generally to employ first order frequencies for aliasing; however, there can be applications where second order 
aliasing has advantages. 

[0122] The general flow diagram for implementation of the copy-deterrent pattern is shown in Fig. 21 . In a Frame 
Selection step 200, n consecutive frames 100 are selected for implementation of the copy-deterrent pattern. Next, in 
a Pattern Selection step 202, specific pixel 1 02 locations within copy-deterrent pattern 1 04 are selected. As mentioned 
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above, pixels 1 02 for modulation may be simply selected at random or may comprise a text or encrypted message. In 
a Pixel Value Calculation step 204, the intensity values of selected pixels 102 (that make up copy-deterrent pattern 
104 and are spanned over n frames 100) are calculated. This calculation may simply entail computation of individual 
R, G and B values or may include computation of an average or hybrid luminance or chrominance measure. In a 

5 Decision step 206, it is made certain that output intensity characteristics/limitations of the projection system do not 
prevent the selected pixels 1 02 from being modulated and projected. If projection is not possible, new pixels 102 must 
be selected. In a Bandwidth Calculation step 208, the bandwidth of the selected pixel intensity pattern (that is spanned 
over n frames) is calculated and/or estimated. As noted earlier, this bandwidth is less than the frame 100 projection 
rate but its exact value depends on the movie content (i.e., number of selected frames 100 and the pixel 102 intensity 

10 values within the selected frames 1 00). Since the number of frames 1 00 and the intensity values of the selected pixels 
102 are precisely known, bandwidth calculations may be carried out by taking the Fourier transform of the selected 
pixels 1 02 and examining its spectrum. However, since this process may require a considerable amount of computation, 
a simple estimate of the bandwidth (e.g., W/2 — 5 Hz for a very conservative estimation) may be used. 
[0123] The next step is a Sampling Frequency Selection step 21 0. This can be selected to be one of several standard 

is sampling rates (e.g., 50, 60, 100, 120 Hz, etc.) used in today's camcorders. It is also possible to select one sampling 
frequency value for the first n., set of-frames 1 00 and select a different sampling rate for the next n2 set of frames 1 00. 
Alternatively, or in conjunction with above, different sampling rates for different regions 108 within the frame 100 may 
be used to carry out the calculations. This way, copy-deterrent pattern 1 04 would affect a wider variety of camcorders. 
In a Modulation Selection step 212, the appropriate modulation scheme (or a combination of them) as is outlined in 

20 (2a) - (2e) above is used at the appropriate modulation frequency to produce aliasing for the chosen sampling rate. In 
order to broaden the effectiveness of this technique, a range of modulation schemes and frequencies may be used to 
affect the same set of pixels 1 02 but in different set of "n M frames 1 00. Finally, once the appropriate modulation scheme 
and frequency is selected, the necessary information is supplied to image-forming assembly 16 in a Modulate step 
21 4 in order to affect the projection of pixels 1 02. 

25 

Apparatus of Preferred Embodiment 

[0124] Referring to Fig. 22, there is shown a block diagram of a copy-deterrent projection apparatus 10 with an 
arrangement of components for implementing the present invention in a preferred embodiment. Image data for pro- 
se jection of each frame 100 is typically provided in compressed form and may be transmitted to the projecting site or 
provided on storage media. In any event, compressed image data is input to a decompression circuit 1 2. Decompression 
circuit 12 decompresses frame 1 00 data and provides this data to a display logic assembly 14. Display logic assembly 
14 formats the image data for frame 100 into pixel array format and provides the necessary color correction, image 
resjzing, and related functions. Then the data is passed to an image forming assembly 16 for display onto a display 
35 screen 20. The decompression and display sequence and apparatus just described are familiar to those skilled in the 
digital motion picture projection arts and may be embodied using a number of different types of components. 
[0125] Pattern data and control parameters are provided to a pattern generator/modulator assembly 1 8 for generating 
and modulating pattern 104. Pattern data and control parameters can be provided from a number of sources. For 
example, pattern data and control parameters can be provided by the motion picture supplier, provided along with the 
40 compressed image data. This arrangement would put the movie supplier in control of pattern 1 04 generation for copy 
protection. At the other extreme, copy protection could be solely in the domain of the projection site itself. In such a 
case, pattern data and control parameters can be provided by optional pattern logic circuitry 22, indicated by a dashed 
box in Fig. 22, or can be generated on the fly at the time of projection, based on known characteristics of the movie 
that must be provided to the projection site. Other possibilities include some combination of control by the movie supplier 
45 and local control by the projection site. 

[0126] The object is to provide the location of pixels 1 02 as well as modulation frequency and intensity variation data. 
As is described previously for methods(la) - (1e) and (2a) - (2f), plaintext or encrypted information can be sent by 
selection of specific location, frequency, and intensity data for modulation of pixels 1 02. Significantly, pattern generator/ 
modulator assembly 1 8 provides this location, frequency, and intensity data for modulation, assigned to existing pixels 
so 1 02 in frame 1 00. That is, pattern generator/modulator assembly 1 8 does not provide "new" pixels 1 02 relative to the 
scene content for frame 1 00. Instead, the output from pattern generator/modulator assembly 1 8 can be considered as 
control signals, sent to image forming assembly 1 6, in order to manipulate the scene content that is sent from display 
logic assembly 14. 

[0127] Pattern generator/modulator assembly 18 provides modulation information for pixels 102 within pattern 104 
55 that are sent to the display logic assembly 14 and the image forming assembly 16 for projection. 

[0128] It is instructive to note that, in addition to image forming assembly 16, decompression circuit 12, and display 
logic assembly 14, pattern generator/modulator assembly 1 8 can be part of projector 1 0 as in the preferred embodiment 
or can be separate components, such as components running on a separate computer or other processor. 
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[0129] Image forming assembly 16 may employ any one of a number of display technologies for projection of a 
sequence of frames 100 onto display screen 20. In a preferred embodiment, image forming assembly 16 comprises a 
transmissive Liquid-Crystal Device (LCD) spatial light modulator and support components for projection of frames 1 00. 
Pattern 104 data may be used to modulate individual pixels within the LCD for each color. Or, a separate LCD may be 
employed for modulation of pattern 1 04. Other types of modulator could alternately be used, including a digital micro- 
mirror device (DMD) or reflective LCDs, for example. 

Alternate techniques for applying modulation: 

[01 30] The apparatus of the present invention can employ any of a number of different techniques for applying mod- 
ulation to pattern 104. Referring to Fig. 23 which shows the signal path that applies for projection of each pixel 102 in 
frame 100, methods for applying modulation include the following: 

(a) Modulation at digital control circuitry. In the preferred embodiment, modulation is applied to a digital signal 30 
within the digital control circuitry used for generation of the array of pixels 1 02 in each frame 1 00. With this method, 
the digital values that correspond to pixels 1 02 in each frame 1 00 are modulated in accordance with the modulation 
scheme. An example of a 50% duty-cycle square wave modulation is depicted in Fig 23 as an "AND" operation 
that can be readily implemented in digital domain. 

(b) Analog modulation. Analog modulation is possible, using some method for modulation of an analog signal 32 
that controls intensity of the light source used for pixels 102 within pattern 104, for example. Analog modulation 
techniques, as opposed to the digital techniques of part a, manipulate continuous-time analog signals. Neverthe- 
less, similar concepts of modulation can be carried over from the digital domain to the analog domain. 

(c) Optical modulation. Optical modulation could be implemented using techniques of projection optics 34 such as 
shuttering, masking, or controlled pixel emission. Optical modulation could be applied within image forming as- 
sembly 16 as well as at display screen 20, using masking or filtering techniques. Referring to Fig. 23, the optical 
modulation mask 216, for example, can be used either in front of the projection optics or on top of the display 
screen to selectively manipulate the projected pixel locations. Liquid crystal display material, with electrically con- 
trolled transmission characteristics, may be used to construct such a spatial light modulation mask; the opacity of 
different regions of such a mask may be controlled by changing the applied electrical signal to that region of the 
mask. 

[0131] As was noted above, modulation can be differently applied for each color component of frame 1 00. 
Alternative Embodiments 

[0132] Referring to Fig. 24, there is shown an alternate embodiment for copy-deterrent projection apparatus 10, in 
which image forming assembly 1 6 does not project onto display screen 20. Instead, image forming assembly 1 6 com- 
municates with an emissive display panel 24. Emissive display panel 24 may be in the form of a panel that comprises 
an LED array 26, similar to the display panels widely used in large sports facilities, for example. Using display panel 
24 of this type, image forming assembly 1 6 directly controls the intensity and modulation of red, green, and blue LEDs, 
28R, 28G, and 28B, respectively. 

[0133] While the invention has been described with particular reference to its preferred embodiments, ft will be un- 
derstood by those skilled in the art that various changes may be made and equivalents may be substituted for elements 
in the preferred embodiments without departing from the scope of the invention. For example, copy-deterrent pattern 
104 can comprise any of a number of arrangements of pixels 102 to form messages 106 or regions 108 anywhere 
within frame 100. Copy-deterrent patterns 104 can be provided primarily in order to obscure a recording or to provide 
digital watermarking, or both. The present invention is adaptable to a number of possible configurations of digital motion 
projector and display screen apparatus, such as using micromirror technology or a light valve array, for example. 
[01 34] Therefore, what is provided is a copy-deterrent projection apparatus for digital motion pictures and a method 
for applying modulation to pixels within a displayed motion picture frame in order to discourage recording of the image 
using a video camera. 

Claims 

1 . A copy-deterrent display apparatus for displaying a sequential plurality of image frames, each of said image frames 
comprising an array of pixels, each said pixel assigned to be displayed at a predetermined intensity within each 
frame, said apparatus comprising: 
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(a) a pattern generator/modulator assembly capable of providing control signals specifying a pattern of pixels 
within each said frame, said control signals specifying temporal modulation of said pattern of pixels, wherein 
said temporal modulation is chosen to be imperceptible to a human observer while simultaneously producing 
objectionable artifacts due to aliasing when said displayed frames are captured by a video capturing device; 

(b) an image forming assembly capable of accepting said control signals from said pattern generator/modulator 
assembly and of modifying the displayed pixel intensities within each said frame, in response to said control 
signals. 

2. The apparatus of claim 1 wherein said modulated pattern of pixels comprises a text message. 

3. The apparatus of claim 1 wherein said modulated pattern of pixels comprises a predetermined copy obscuration 
pattern. 

4. The apparatus of claim 1 wherein said modulated pattern of pixels comprises a random pattern. 

5. The apparatus of any of claims 1-4 and wherein in order to provide copy deterrence to a video capturing device 
having a sampling frequency of 60 Hertz and/or 120 Hertz a temporal modulation frequency is used that meets 
the criteria of the absolute value of the difference between the sampling frequency and the temporal modulation 
frequency is greater than or equal to 10 Hertz and less than or equal to 30 Hertz. 

6. A method for displaying a copy deterrent pattern in a sequential plurality of image frames, each of said image 
frames comprising an array of pixels, said pattern comprising a plurality of pixels selected from said frame, the 
method comprising temporally modulating said pattern wherein said temporal modulation is chosen to be imper- 
ceptible to a human observer while simultaneously producing objectionable artifacts due to aliasing when said 
displayed frames are captured by a video capturing device. 

7. The method of claim 6 wherein said copy deterrent pattern comprises a message. 

8. The method of claim 6 wherein said copy deterrent pattern comprises a predetermined copy obscuration pattern. 

9. The method of claim 6 wherein said copy deterrent pattern comprises a random pattern. 

10. The method of claim 6 wherein said copy deterrent pattern comprises a digital watermark. 

11. The method of claim 6 wherein said copy deterrent pattern substantially comprises said frame. 

12. The method of claim 6 wherein the plurality of image frames form part of a digital movie or video image having a 
predetermined refresh rate and wherein said pattern is temporally modulated at af requency higherthan the refresh 
rate. 

13. The method of claim 12 and wherein the temporal modulation is sinusoidal. 

14. The method of claim 13 and wherein intensity of the pixels comprising the modulated pattern is adjusted to com- 
pensate for the modulation. 

15. The method of claim 6 or 12 and wherein the modulation is a rectangular wave. 

16. The method of claim 6 or 12 and wherein one or more color components of an image in the frame is subject to 
said temporal modulation but other color components of the same image frame are not subject to said temporal 
modulation. 

17. The method according to claim 6 or 12 and wherein the pattern comprises plural portions and the plural portions 
are temporarily modulated at different frequencies from each other. 

18. The method according to claim 6 or 12 and wherein the frequency of modulation is changed from one frequency 
to another frequency during the course of display. 

1 9. The method according to claim 6 or 1 2 and wherein the plurality of pixels are analyzed according to a peak intensity 
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criterion and pixels meeting the criterion further determine the pattern that is subject to the temporal modulation. 
20. The method of claim 6 or 12 and wherein the temporal modulation is sinusoidal. 

21 The method according to any of claims 6 through 20 and wherein in order to provide copy deterrence to a video 
" capturing device having a sampling frequency of 60 Hertz and/or 120 Hertz a temporal modulation frequency is 

used that meets the criteria of the absolute val ue of the difference between the sampling frequency and the temporal 
modulation frequency is greater than or equal to 10 Hertz and less than or equal to 30 Hertz. 

22 A method for identifying a candidate modulation frequency to be applied to a selected pattern of pixels within a 
* sequence of digital motion picture frames, said candidate modulation frequency intended to cause aliasing when 

the sequence of digital motion picture frames is sampled using a video capture device, said method comprising: 

(a) selecting a plurality of sequential frames; 

(b) obtaining intensity values for each pixel within said selected pattern of pixels; 

(c) calculating a modulation intensity value for each said pixel within said selected pattern of pixels, said mod- 
ulation intensity value conditioned by the duty cycle for a modulation signal waveform; 

(d) calculating a bandwidth for said pixels within said selected pattern of pixels; 

(e) choosing a target sampling frequency; and 

(f) calculating said candidate modulation frequency based on said target sampling frequency and based on 
predetermined thresholds for visible modulation, said candidate modulation frequency selected based on hav- 
ing a first order or a second order side band within bounds of said predetermined thresholds for visible mod- 
ulation. 
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TABLE 2. SPECTRAL SPREAD FOR MAIN, LEFT AND RIGHT 
FREQUENCY BANDS FOR W/2=30 HERTZ ANDfs=120 Hz 
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TABLE 3. SPECTRAL SPREAD FOR MAIN, LEFT AND RIGHT 
FREQUENCY BANDS FOR W/2=24 HERTZ AND£s=120 Hz 
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EP 1 237 369 A2 



TABLE 5. SPECTRAL SPREAD FOR MAIN, LEFT AND RIGHT 
FREQUENCY BANDS FOR W/2=5 HERTZ AND fs=120 H2 



W/2 



LEFT 
BAND 
£m FROM TO 



fs 120 

1ST 
SPECTRAL 
ORDER 
MAIN 
BAND 
"ROM TO 



50 

55 

60 

65 

70 

75 

80 

85 

90 

95 

100 

105 

110 

115 

120 

125 

130 

135 

140 

145 

150 

155 

160 

165 

170 

175 

180 

185 

190 

195 

200 

205 

210 



65 
60 
55 
50 
45 
40 
35 
30 
25 
20 
15 
10 

5 

0 

-5 
-.10 
-15 
-20 
-25 
-30 
-35 
-40 
-45 
-50 
-55 
-60 
-65 
-70 
-75 
-80 
-85 
-90 
-95 



75 

70 

65 

60 

55 

50 

45 

40 

35 

30 

25 

20 

15 

10 

5 

0 

-5 

-10 

-15 

-20 

-25 

-30 

-35 

-40 

-45 

-50 

-55 

-60 

-65 

-70 

-75 

-80 

-85 



115 

115 

115 

115 

115 

115 

115 

115 

115 

115 

115 

115 

115 

115 

115 

115 

115 

115 

115 

115 

115 

115 

115 

115 

115 

115 

115 

115 

115 

115 

115 

115 

115 



125 

125 

125 

125 

125 

125 

125 

125 

125 

125 

125 

125 

125 

125 

125 

125 

125 

125 

125 

125 

125 

125 

125 

125 

125 

125 

125 

125 

125 

125 

125 

125 

125 



RIGHT 
BAND 
ROM TO 



165 
170 
175 
180 
185 



175 
180 
185 
190 
195 



190 200 
195 205 
200 210 
205 215 
210 220 
215 225 
220 230 
225 235 
230 240 
235 245 
240 250 
245 255 
250 260 
255 265 
260 270 
265 275 
270 280 
275 285 
280 290 
285 295 
290 300 
295 305 
300 310 
305 315 
310 320 
315 325 
320 330 
325 335 



LEFT 
BAND 
FROM TO 



2ND 
SPECTRAL 
ORDER 
MAIN 
BAND 



RIGHT 
BAND 



FROM TO FROM TO 



185 
180 
175 
170 
165 
160 
155 
150 
145 
140 
135 
130 
125 
120 
115 
110 
105 
100 
95 
90 
85 
80 
75 
70 
65 
60 
55 
50 
45 
40 
35 
30 
25 



195 
190 
185 
180 
175 
170 
165 
160 
155 
150 
145 
140 
135 
130 
125 
120 
115 
110 
105 
100 
95 
90 
85 
80 
75 
70 
65 
60 
55 
50 
45 
40 
35 



235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 
235 245 



285 295 
290 300 
295 305 
300 310 
305 315 
310 320 
315 325 
320 330 
325 335 
330 340 
335 345 
340 350 
345 355 
350 360 
355 365 
360 370 
365 375 
370 380 
375 385 
380 390 
385 395 
390 400 
395 405 
400 410 
405 415 
410 420 
415 425 
420 430 
425 435 
430 440 
435 445 
440 450 
445 455 
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